# -------------- IR-based methods -------------- #
Bridging the KB-Text Gap: Leveraging Structured Knowledge-aware Pre-training for KBQA 
摘要:知识库问答(Knowledge Base Question Answering, KBQA)旨在回答含有知识库中实体、关系等事实信息的自然语言问题。然而，传统的预训练语言模型(plm)是直接在大规模自然语言语料库上进行预训练的，这给它们理解和表示结构化KBs中的复杂子图带来了挑战。为弥合文本和结构化KBs之间的差距，本文提出一种结构化知识感知预训练方法(SKP)。在预训练阶段，引入了两个新的结构化知识感知任务，指导模型有效地学习隐式关系和复杂子图的更好表示。在下游KBQA任务中，进一步设计了高效的线性化策略和区间注意力机制，分别辅助模型在推理过程中更好地编码复杂子图和屏蔽不相关子图的干扰。在WebQSP上进行了详细的实验和分析，验证了SKP的有效性，特别是在子图检索方面的显著改进(+4.08\% H@10)。
@inproceedings{Dong-Guanting-CIKM-2023-Bridging-the-KB-Text-Gap,
    author = {Dong, Guanting and Li, Rumei and Wang, Sirui and Zhang, Yupeng and Xian, Yunsen and Xu, Weiran},
    title = {Bridging the KB-Text Gap: Leveraging Structured Knowledge-aware Pre-training for KBQA},
    year = {2023},
    isbn = {9798400701245},
    publisher = {Association for Computing Machinery},
    address = {New York, NY, USA},
    url = {https://doi.org/10.1145/3583780.3615150},
    doi = {10.1145/3583780.3615150},
    abstract = {Knowledge Base Question Answering (KBQA) aims to answer natural language questions with factual information such as entities and relations in KBs. However, traditional Pre-trained Language Models (PLMs) are directly pre-trained on large-scale natural language corpus, which poses challenges for them in understanding and representing complex subgraphs in structured KBs. To bridge the gap between texts and structured KBs, we propose a Structured Knowledge-aware Pre-training method (SKP). In the pre-training stage, we introduce two novel structured knowledge-aware tasks, guiding the model to effectively learn the implicit relationship and better representations of complex subgraphs. In downstream KBQA task, we further design an efficient linearization strategy and an interval attention mechanism, which assist the model to better encode complex subgraphs and shield the interference of irrelevant subgraphs during reasoning respectively. Detailed experiments and analyses on WebQSP verify the effectiveness of SKP, especially the significant improvement in subgraph retrieval (+4.08\% H@10).},
    booktitle = {Proceedings of the 32nd ACM International Conference on Information and Knowledge Management},
    pages = {3854–3859},
    numpages = {6},
    keywords = {structured knowledge, pre-training, kbqa, efficient linearization},
    location = {<conf-loc>, <city>Birmingham</city>, <country>United Kingdom</country>, </conf-loc>},
    series = {CIKM '23}
}

Subgraph retrieval enhanced model for multi-hop knowledge base question answering 
最近关于知识库问答(KBQA)的工作检索子图以方便推理。所需的子图至关重要，因为小的子图可能会排除答案，而大的子图可能会引入更多的噪声。然而，现有的检索要么是启发式的，要么与推理交织在一起，导致在部分子图上进行推理，在中间监督缺失时增加了推理偏差。本文提出了一种与后续推理过程解耦的可训练子图检索器(SR)，它使即插即用框架能够增强任何面向子图的KBQA模型。实验结果表明，与现有的检索方法相比，SR具有更好的检索和问答性能。通过弱监督预训练和端到端微调，SR与NSM (He等人，2021)相结合，用于基于嵌入的KBQA方法，实现了新的最先进的性能。NSM是一种面向子图的推理机。
@inproceedings{Zhang-Jing-ACL-2022-Subgraph-Retrieval-Enhanced-Model,
    title = "Subgraph Retrieval Enhanced Model for Multi-hop Knowledge Base Question Answering",
    author = "Zhang, Jing  and
      Zhang, Xiaokang  and
      Yu, Jifan  and
      Tang, Jian  and
      Tang, Jie  and
      Li, Cuiping  and
      Chen, Hong",
    editor = "Muresan, Smaranda  and
      Nakov, Preslav  and
      Villavicencio, Aline",
    booktitle = "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = may,
    year = "2022",
    address = "Dublin, Ireland",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.acl-long.396",
    doi = "10.18653/v1/2022.acl-long.396",
    pages = "5773--5784",
    abstract = "Recent works on knowledge base question answering (KBQA) retrieve subgraphs for easier reasoning. The desired subgraph is crucial as a small one may exclude the answer but a large one might introduce more noises. However, the existing retrieval is either heuristic or interwoven with the reasoning, causing reasoning on the partial subgraphs, which increases the reasoning bias when the intermediate supervision is missing. This paper proposes a trainable subgraph retriever (SR) decoupled from the subsequent reasoning process, which enables a plug-and-play framework to enhance any subgraph-oriented KBQA model. Extensive experiments demonstrate SR achieves significantly better retrieval and QA performance than existing retrieval methods. Via weakly supervised pre-training as well as the end-to-end fine-tuning, SR achieves new state-of-the-art performance when combined with NSM (He et al., 2021), a subgraph-oriented reasoner, for embedding-based KBQA methods. Codes and datasets are available online (\url{https://github.com/RUCKBReasoning/SubgraphRetrievalKBQA})",
}


Large-scale relation learning for question answering over knowledge bases with pre-trained language models
知识库问答(KBQA)的关键挑战是自然语言问题与知识库(KB)中推理路径的不一致性。目前基于图的KBQA方法善于把握图的拓扑结构，但往往忽略了节点和边所携带的文本信息。同时，预训练语言模型从大规模语料库中学习大量的开放世界知识，但这些知识是自然语言形式，没有结构。为了弥合自然语言和结构化知识库之间的鸿沟，该文提出了基于bert的KBQA中的3个关系学习任务，包括关系抽取、关系匹配和关系推理。通过关系增强训练，该模型学会将自然语言表达与知识库中的关系对齐，并对知识库中缺失的连接进行推理。在WebQSP数据集上的实验表明，该方法优于其他基线方法，特别是在知识库不完整的情况下。
@inproceedings{Yan-Yuanmeng-EMNLP-2021-Large-scale-relation-learning,
    title = "Large-Scale Relation Learning for Question Answering over Knowledge Bases with Pre-trained Language Models",
    author = "Yan, Yuanmeng  and
      Li, Rumei  and
      Wang, Sirui  and
      Zhang, Hongzhi  and
      Daoguang, Zan  and
      Zhang, Fuzheng  and
      Wu, Wei  and
      Xu, Weiran",
    editor = "Moens, Marie-Francine  and
      Huang, Xuanjing  and
      Specia, Lucia  and
      Yih, Scott Wen-tau",
    booktitle = "Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2021",
    address = "Online and Punta Cana, Dominican Republic",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.emnlp-main.296",
    doi = "10.18653/v1/2021.emnlp-main.296",
    pages = "3653--3660",
    abstract = "The key challenge of question answering over knowledge bases (KBQA) is the inconsistency between the natural language questions and the reasoning paths in the knowledge base (KB). Recent graph-based KBQA methods are good at grasping the topological structure of the graph but often ignore the textual information carried by the nodes and edges. Meanwhile, pre-trained language models learn massive open-world knowledge from the large corpus, but it is in the natural language form and not structured. To bridge the gap between the natural language and the structured KB, we propose three relation learning tasks for BERT-based KBQA, including relation extraction, relation matching, and relation reasoning. By relation-augmented training, the model learns to align the natural language expressions to the relations in the KB as well as reason over the missing connections in the KB. Experiments on WebQSP show that our method consistently outperforms other baselines, especially when the KB is incomplete.",
}

TransferNet: An effective and transparent framework for multi-hop question answering over relation graph.
摘要= "多跳问答(QA)是一项具有挑战性的任务，因为它需要在回答的每一步都进行精确的实体关系推理。这种关系可以用知识图谱中的标签(如配偶)或文本语料库中的文本(如结婚26年)来表示。现有模型通常通过预测顺序关系路径或聚合隐藏图特征来推断答案。前者难以优化，后者缺乏可解释性。该文提出了TransferNet，一种高效透明的多跳问答模型，在统一的框架下同时支持标签和文本关系。TransferNet在多个步骤中跨越实体。在每一步，它关注问题的不同部分，计算关系的激活分数，然后以可微的方式沿激活的关系迁移先前的实体分数。在三个数据集上进行了广泛的实验，证明TransferNet在很大程度上超过了最先进的模型。特别地，在MetaQA上，它在2-hop和3-hop问题上都达到了100{\%}的准确率。通过定性分析，我们表明TransferNet具有透明和可解释的中间结果。
@inproceedings{Shi-Jiaxin-EMNLP-2021-TransferNet,
    title = "{T}ransfer{N}et: An Effective and Transparent Framework for Multi-hop Question Answering over Relation Graph",
    author = "Shi, Jiaxin  and
      Cao, Shulin  and
      Hou, Lei  and
      Li, Juanzi  and
      Zhang, Hanwang",
    editor = "Moens, Marie-Francine  and
      Huang, Xuanjing  and
      Specia, Lucia  and
      Yih, Scott Wen-tau",
    booktitle = "Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2021",
    address = "Online and Punta Cana, Dominican Republic",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.emnlp-main.341",
    doi = "10.18653/v1/2021.emnlp-main.341",
    pages = "4149--4158",
    abstract = "Multi-hop Question Answering (QA) is a challenging task because it requires precise reasoning with entity relations at every step towards the answer. The relations can be represented in terms of labels in knowledge graph (e.g., spouse) or text in text corpus (e.g., they have been married for 26 years). Existing models usually infer the answer by predicting the sequential relation path or aggregating the hidden graph features. The former is hard to optimize, and the latter lacks interpretability. In this paper, we propose TransferNet, an effective and transparent model for multi-hop QA, which supports both label and text relations in a unified framework. TransferNet jumps across entities at multiple steps. At each step, it attends to different parts of the question, computes activated scores for relations, and then transfer the previous entity scores along activated relations in a differentiable way. We carry out extensive experiments on three datasets and demonstrate that TransferNet surpasses the state-of-the-art models by a large margin. In particular, on MetaQA, it achieves 100{\%} accuracy in 2-hop and 3-hop questions. By qualitative analysis, we show that TransferNet has transparent and interpretable intermediate results.",
}

Improving multi-hop knowledge base question answering by learning intermediate supervision signals.
多跳知识库问答(KBQA)旨在找出知识库(KB)中与问题实体相距多跳的答案实体。一个主要挑战是中间步骤缺乏监督信号。因此，多跳KBQA算法只能接收最终答案的反馈，导致学习不稳定或无效。为应对这一挑战，本文提出一种新的教师-学生方法用于多跳KBQA任务。在我们的方法中，学生网络旨在找到查询的正确答案，而教师网络试图学习中间监督信号，以提高学生网络的推理能力。主要的新颖性在于教师网络的设计，利用前向和后向推理来增强中间实体分布的学习。通过考虑双向推理，教师网络可以产生更可靠的中间监督信号，可以缓解虚假推理问题。在三个基准数据集上的广泛实验证明了所提方法在KBQA任务上的有效性。
@inproceedings{HeGaole-ICWSM-2021-Improving-Multi-hop-Knowledge-Base-Question-Answering,
    author = {He, Gaole and Lan, Yunshi and Jiang, Jing and Zhao, Wayne Xin and Wen, Ji-Rong},
    title = {Improving Multi-hop Knowledge Base Question Answering by Learning Intermediate Supervision Signals},
    year = {2021},
    isbn = {9781450382977},
    publisher = {Association for Computing Machinery},
    address = {New York, NY, USA},
    url = {https://doi.org/10.1145/3437963.3441753},
    doi = {10.1145/3437963.3441753},
    abstract = {Multi-hop Knowledge Base Question Answering (KBQA) aims to find the answer entities that are multiple hops away in the Knowl- edge Base (KB) from the entities in the question. A major challenge is the lack of supervision signals at intermediate steps. Therefore, multi-hop KBQA algorithms can only receive the feedback from the final answer, which makes the learning unstable or ineffective. To address this challenge, we propose a novel teacher-student approach for the multi-hop KBQA task. In our approach, the stu- dent network aims to find the correct answer to the query, while the teacher network tries to learn intermediate supervision signals for improving the reasoning capacity of the student network. The major novelty lies in the design of the teacher network, where we utilize both forward and backward reasoning to enhance the learning of intermediate entity distributions. By considering bidi- rectional reasoning, the teacher network can produce more reliable intermediate supervision signals, which can alleviate the issue of spurious reasoning. Extensive experiments on three benchmark datasets have demonstrated the effectiveness of our approach on the KBQA task.},
    booktitle = {Proceedings of the 14th ACM International Conference on Web Search and Data Mining},
    pages = {553–561},
    numpages = {9},
    keywords = {teacher-student network, knowledge base question answering, intermediate supervision signals},
    location = {Virtual Event, Israel},
    series = {WSDM '21}
}
# -------------- LLM+IR -------------- #
Structgpt: A general framework for large language model to reason over structured data
abstract = "本文旨在以统一的方式提高大型语言模型(LLMs)对结构化数据的推理能力。受llm工具增强研究的启发，本文开发了一种迭代阅读-然后推理(IRR)框架，以解决基于结构化数据的问答任务，称为StructGPT。在该框架中，构建了专门的接口来从结构化数据(即阅读)中收集相关证据，并让llm专注于基于收集的信息的推理任务(即推理)。特别地，我们提出了一个调用线性化生成过程，以支持llm借助接口对结构化数据进行推理。通过使用提供的接口迭代该过程，该方法可以逐渐接近给定查询的目标答案。在三种结构化数据上进行的实验表明，在少样本和零样本设置下，StructGPT极大地提高了LLMs的性能。
@inproceedings{Jiang-EMNLP-2023-StructGPT,
    title = "{S}truct{GPT}: A General Framework for Large Language Model to Reason over Structured Data",
    author = "Jiang, Jinhao  and
      Zhou, Kun  and
      Dong, Zican  and
      Ye, Keming  and
      Zhao, Xin  and
      Wen, Ji-Rong",
    editor = "Bouamor, Houda  and
      Pino, Juan  and
      Bali, Kalika",
    booktitle = "Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing",
    month = dec,
    year = "2023",
    address = "Singapore",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.emnlp-main.574",
    doi = "10.18653/v1/2023.emnlp-main.574",
    pages = "9237--9251",
    abstract = "In this paper, we aim to improve the reasoning ability of large language models (LLMs) over structured data in a unified way. Inspired by the studies on tool augmentation for LLMs, we develop an Iterative Reading-then-Reasoning (IRR) framework to solve question answering tasks based on structured data, called StructGPT. In this framework, we construct the specialized interfaces to collect relevant evidence from structured data (i.e., reading), and let LLMs concentrate on the reasoning task based on the collected information (i.e., reasoning). Specially, we propose an invoking-linearization-generation procedure to support LLMs in reasoning on the structured data with the help of the interfaces. By iterating this procedure with provided interfaces, our approach can gradually approach the target answers to a given query. Experiments conducted on three types of structured data show that StructGPT greatly improves the performance of LLMs, under the few-shot and zero-shot settings.",
}

Think-on-graph: Deep and responsible reasoning of large language model on knowledge graph
摘要= "尽管大型语言模型(llm)在各种任务中取得了重大成功，但它们经常与幻觉问题作斗争，特别是在需要深度和负责任推理的场景中。这些问题可以通过在LLM推理中引入外部知识图谱(KG)来部分解决。该文提出了一种新的LLM-KG集成范式" LLM x KG "，将LLM视为一个agent，交互式地探索知识图谱上的相关实体和关系，并根据检索到的知识进行推理。通过引入一种名为Think-on-Graph (ToG)的新方法进一步实现了这种范式，其中LLM智能体在KG上迭代地执行波束搜索，发现最有希望的推理路径，并返回最有可能的推理结果。使用大量精心设计的实验来检验和说明ToG的以下优点:1)与llm相比，ToG具有更好的深度推理能力;2)借助LLMs推理和专家反馈，ToG具有知识追溯和知识修正能力;3) ToG为不同的llm、知识经理和激励策略提供了一个灵活的即插即用框架，而无需任何额外的培训成本;4)在某些场景下，使用小型LLM模型的ToG的性能可能超过GPT-4等大型LLM，这降低了LLM的部署和应用成本。作为一种计算成本较低和通用性较好的无训练方法，ToG在9个数据集中的6个数据集中实现了总体SOTA，其中大多数以前的SOTA依赖于额外的训练。”
@inproceedings{Jiashuo-Sun-ICLR-2024-Think-on-Graph,
    title={Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph},
    author={Jiashuo Sun and Chengjin Xu and Lumingyuan Tang and Saizhuo Wang and Chen Lin and Yeyun Gong and Heung-Yeung Shum and Jian Guo},
    booktitle={The Twelfth International Conference on Learning Representations},
    year={2024},
    url={https://openreview.net/forum?id=nnVO1PvbTv},
    abstract = "Although large language models (LLMs) have achieved significant success in various tasks, they often struggle with hallucination problems, especially in scenarios requiring deep and responsible reasoning. These issues could be partially addressed by introducing external knowledge graphs (KG) in LLM reasoning. In this paper, we propose a new LLM-KG integrating paradigm ``LLM x KG'' which treats the LLM as an agent to interactively explore related entities and relations on KGs and perform reasoning based on the retrieved knowledge. We further implement this paradigm by introducing a new approach called Think-on-Graph (ToG), in which the LLM agent iteratively executes beam search on KG, discovers the most promising reasoning paths, and returns the most likely reasoning results. We use a number of well-designed experiments to examine and illustrate the following advantages of ToG: 1) compared with LLMs, ToG has better deep reasoning power; 2) ToG has the ability of knowledge traceability and knowledge correctability by leveraging LLMs reasoning and expert feedback; 3) ToG provides a flexible plug-and-play framework for different LLMs, KGs and prompting strategies without any additional training cost; 4) the performance of ToG with small LLM models could exceed large LLM such as GPT-4 in certain scenarios and this reduces the cost of LLM deployment and application. As a training-free method with lower computational cost and better generality, ToG achieves overall SOTA in 6 out of 9 datasets where most previous SOTAs rely on additional training."
}

# -------------- Semantic Parsing-based methods -------------- #
Outlining and filling: Hierarchical query graph generation for answering complex questions over knowledge graphs
摘要={查询图构造的目的是在知识图谱上构造正确的可执行SPARQL来回答自然语言问题。尽管最近的方法在基于神经网络的查询图排序方面取得了良好的效果，但它们在处理更复杂的问题时面临着3个新的挑战:1)复杂的SPARQL语法，2)巨大的搜索空间，3)局部模糊的查询图。本文提出了一种新的解决方案。作为准备，我们将查询图的每个SPARQL子句看作由边和顶点组成的子图，并定义了一个统一的图语法AQG来描述查询图的结构。基于这些概念，本文提出了一种新的端到端模型，通过执行分层自回归解码来生成查询图。高层解码生成一个AQG作为约束，以剪枝搜索空间并减少局部二义查询图。底层解码通过从预准备的候选实例中选择合适的实例填充AQG中的槽位来完成查询图的构建。实验结果表明，该方法极大地提高了SOTA在复杂KGQA测试集上的性能。在配备了预训练模型的情况下，所提出方法的性能得到了进一步提高，对使用的所有三个数据集都实现了SOTA。
@article{Chen-Yongrui-TKDE-2023-Outlining-and-Filling,
    author = {Chen, Yongrui and Li, Huiying and Qi, Guilin and Wu, Tianxing and Wang, Tenggou},
    title = {Outlining and Filling: Hierarchical Query Graph Generation for Answering Complex Questions Over Knowledge Graphs},
    year = {2023},
    issue_date = {Aug. 2023},
    publisher = {IEEE Educational Activities Department},
    address = {USA},
    volume = {35},
    number = {8},
    issn = {1041-4347},
    url = {https://doi.org/10.1109/TKDE.2022.3207477},
    doi = {10.1109/TKDE.2022.3207477},
    abstract = {Query graph construction aims to construct the correct executable SPARQL on the KG to answer natural language questions. Although recent methods have achieved good results using neural network-based query graph ranking, they suffer from three new challenges when handling more complex questions: 1) complicated SPARQL syntax, 2) huge search space, and 3) locally ambiguous query graphs. In this paper, we provide a new solution. As a preparation, we extend the query graph by treating each SPARQL clause as a subgraph consisting of vertices and edges and define a unified graph grammar called AQG to describe the structure of query graphs. Based on these concepts, we propose a novel end-to-end model that performs hierarchical autoregressive decoding to generate query graphs. The high-level decoding generates an AQG as a constraint to prune the search space and reduce the locally ambiguous query graph. The bottom-level decoding accomplishes the query graph construction by selecting appropriate instances from the preprepared candidates to fill the slots in the AQG. The experimental results show that our method greatly improves the SOTA performance on complex KGQA benchmarks. Equipped with pre-trained models, the performance of our method is further improved, achieving SOTA for all three datasets used.},
    journal = {IEEE Trans. on Knowl. and Data Eng.},
    month = {aug},
    pages = {8343–8357},
    numpages = {15}
}

Beamqa: Multi-hop knowledge graph question answering with sequence-to-sequence prediction and beam search.
知识图谱问答(Knowledge Graph Question Answering, KGQA)是一项旨在通过从知识图谱中抽取事实来回答自然语言查询的任务。目前最先进的KGQA技术依赖于来自图实体和关系标签的文本信息，以及外部文本语料库。通过对图中的多个边进行推理，可以准确地排序并返回最相关的实体。然而，这些方法的局限性之一是无法处理现实世界知识图谱固有的不完整性，并且可能因边的缺失而导致答案不准确。为了解决这个问题，图表示学习的最新进展导致了系统的开发，这些系统可以使用链路预测技术来概率地处理缺失的边，使系统能够在不完整的信息下进行推理。然而，使用此类技术的现有KGQA框架往往依赖于学习从查询表示到图嵌入空间的转换，这需要访问大型训练数据集。本文提出BeamQA，一种通过将序列到序列预测模型与嵌入空间中的波束搜索执行相结合的方法，克服了这些限制。该模型使用了预训练的大型语言模型和合成问题生成。在两个知识图谱问答数据集上与其他KGQA方法进行比较，实验证明了BeamQA的有效性。
@inproceedings{Atif-Farah-SIGIR-2023-BeamQA,
    author = {Atif, Farah and El Khatib, Ola and Difallah, Djellel},
    title = {BeamQA: Multi-hop Knowledge Graph Question Answering with Sequence-to-Sequence Prediction and Beam Search},
    year = {2023},
    isbn = {9781450394086},
    publisher = {Association for Computing Machinery},
    address = {New York, NY, USA},
    url = {https://doi.org/10.1145/3539618.3591698},
    doi = {10.1145/3539618.3591698},
    abstract = {Knowledge Graph Question Answering (KGQA) is a task that aims to answer natural language queries by extracting facts from a knowledge graph. Current state-of-the-art techniques for KGQA rely on text-based information from graph entity and relations labels, as well as external textual corpora. By reasoning over multiple edges in the graph, these can accurately rank and return the most relevant entities. However, one of the limitations of these methods is that they cannot handle the inherent incompleteness of real-world knowledge graphs and may lead to inaccurate answers due to missing edges. To address this issue, recent advances in graph representation learning have led to the development of systems that can use link prediction techniques to handle missing edges probabilistically, allowing the system to reason with incomplete information. However, existing KGQA frameworks that use such techniques often depend on learning a transformation from the query representation to the graph embedding space, which requires access to a large training dataset. We present BeamQA, an approach that overcomes these limitations by combining a sequence-to-sequence prediction model with beam search execution in the embedding space. Our model uses a pre-trained large language model and synthetic question generation. Our experiments demonstrate the effectiveness of BeamQA when compared to other KGQA methods on two knowledge graph question-answering datasets.},
    booktitle = {Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval},
    pages = {781–790},
    numpages = {10},
    keywords = {knowledge graphs, question answering},
    location = {<conf-loc>, <city>Taipei</city>, <country>Taiwan</country>, </conf-loc>},
    series = {SIGIR '23}
}

UniKGQA: Unified retrieval and reasoning for solving multi-hop question answering over knowledge graph
摘要= "知识图谱上的多跳问答(KGQA)旨在在大规模的知识图谱(KG)上寻找与自然语言问题中所提到的主题实体具有多跳距离的答案实体。为了应对庞大的搜索空间，现有工作通常采用两阶段的方法:首先检索与问题相关的相对较小的子图，然后在子图上进行推理以准确地找到答案实体。虽然这两个阶段高度相关，但以往的工作在构建检索和推理模型时采用了截然不同的技术解决方案，忽略了它们在任务本质上的关联性。本文提出UniKGQA，一种用于多跳KGQA任务的新方法，通过统一模型架构和参数学习的检索和推理。在模型结构上，UniKGQA由基于预训练语言模型的语义匹配模块(PLM)和匹配信息传播模块(PLM)组成，前者用于实现问题-关系语义匹配，后者用于实现匹配信息在知识图谱上的有向边传播。与以往的研究相比，该方法更加统一，检索和推理阶段联系更紧密。在三个基准数据集上的广泛实验，证明了所提出方法在多跳KGQA任务上的有效性。”
@inproceedings{Jinhao-Jiang-ICLR-2023-UniKGQA,
    title={Uni{KGQA}: Unified Retrieval and Reasoning for Solving Multi-hop Question Answering Over Knowledge Graph},
    author={Jinhao Jiang and Kun Zhou and Xin Zhao and Ji-Rong Wen},
    booktitle={The Eleventh International Conference on Learning Representations },
    year={2023},
    url={https://openreview.net/forum?id=Z63RvyAZ2Vh},
    abstract = "Multi-hop Question Answering over Knowledge Graph~(KGQA) aims to find the answer entities that are multiple hops away from the topic entities mentioned in a natural language question on a large-scale Knowledge Graph (KG). To cope with the vast search space, existing work usually adopts a two-stage approach: it first retrieves a relatively small subgraph related to the question and then performs the reasoning on the subgraph to find the answer entities accurately. Although these two stages are highly related, previous work employs very different technical solutions for developing the retrieval and reasoning models, neglecting their relatedness in task essence.  In this paper, we propose UniKGQA, a novel approach for multi-hop KGQA task, by unifying  retrieval and reasoning in both model architecture and parameter learning. For model architecture, UniKGQA consists of a semantic matching module based on a pre-trained language model~(PLM) for question-relation semantic matching, and a matching information propagation module to propagate the matching information along the directed edges on KGs. For parameter learning, we design a shared pre-training task based on question-relation matching for both retrieval and reasoning models, and then propose retrieval- and reasoning-oriented fine-tuning strategies. Compared with previous studies, our approach is more unified, tightly relating the retrieval and reasoning stages.  Extensive experiments on three benchmark datasets have demonstrated the effectiveness of our method on the multi-hop KGQA task."
}

# -------------- SP - 生成式模型 -------------- #
ReTraCk: A flexible and efficient framework for knowledge base question answering.
摘要= "本文提出了retriver - transducer - checker (ReTraCk)，一种用于大规模知识库问答(KBQA)的神经语义解析框架。ReTraCk被设计为模块化框架以保持高灵活性。它包括一个高效检索相关知识库项的检索器、一个生成语法正确性保证的逻辑形式的转换器和一个改进转导过程的检查器。ReTraCk在GrailQA排行榜上的整体性能排名top1，并在典型的WebQuestionsSP基准上获得了极具竞争力的性能。我们的系统可以及时与用户交互，证明了所提出框架的有效性。”
@inproceedings{Chen-Shuang-ACL-2021-ReTraCk,
    title = "{R}e{T}ra{C}k: A Flexible and Efficient Framework for Knowledge Base Question Answering",
    author = "Chen, Shuang  and
      Liu, Qian  and
      Yu, Zhiwei  and
      Lin, Chin-Yew  and
      Lou, Jian-Guang  and
      Jiang, Feng",
    editor = "Ji, Heng  and
      Park, Jong C.  and
      Xia, Rui",
    booktitle = "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations",
    month = aug,
    year = "2021",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.acl-demo.39",
    doi = "10.18653/v1/2021.acl-demo.39",
    pages = "325--336",
    abstract = "We present Retriever-Transducer-Checker (ReTraCk), a neural semantic parsing framework for large scale knowledge base question answering (KBQA). ReTraCk is designed as a modular framework to maintain high flexibility. It includes a retriever to retrieve relevant KB items efficiently, a transducer to generate logical form with syntax correctness guarantees and a checker to improve transduction procedure. ReTraCk is ranked at top1 overall performance on the GrailQA leaderboard and obtains highly competitive performance on the typical WebQuestionsSP benchmark. Our system can interact with users timely, demonstrating the efficiency of the proposed framework.",
}

Case-based reasoning for natural language queries over knowledge bases.
摘要=“从头开始解决一个复杂的问题通常是具有挑战性的，但如果我们可以使用它们的解决方案访问其他类似的问题，则会容易得多——一种称为基于案例的推理(CBR)的范式。本文提出了一种用于大型知识库问答的神经符号CBR方法(CBR- kbqa)。CBR-KBQA由一个存储案例(问题和逻辑形式)的非参数记忆和一个参数模型组成，参数模型可以通过检索与新问题相关的案例来生成新问题的逻辑形式。在几个包含复杂问题的KBQA数据集上，CBR-KBQA取得了有竞争力的性能。例如，在CWQ数据集上，CBR-KBQA比当前最先进的方法在准确率上高出11％。此外，CBR-KBQA能够使用新的案例{}{}\textit{没有}任何进一步的训练:通过在案例记忆中合并一些人工标记的示例，CBR-KBQA能够成功地生成包含未见过的KB实体和关系的逻辑形式。
@inproceedings{Das-Rajarshi-EMNLP-2021-Case-based-Reasoning,
    title = "Case-based Reasoning for Natural Language Queries over Knowledge Bases",
    author = "Das, Rajarshi  and
      Zaheer, Manzil  and
      Thai, Dung  and
      Godbole, Ameya  and
      Perez, Ethan  and
      Lee, Jay Yoon  and
      Tan, Lizhen  and
      Polymenakos, Lazaros  and
      McCallum, Andrew",
    editor = "Moens, Marie-Francine  and
      Huang, Xuanjing  and
      Specia, Lucia  and
      Yih, Scott Wen-tau",
    booktitle = "Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2021",
    address = "Online and Punta Cana, Dominican Republic",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.emnlp-main.755",
    doi = "10.18653/v1/2021.emnlp-main.755",
    pages = "9594--9611",
    abstract = "It is often challenging to solve a complex problem from scratch, but much easier if we can access other similar problems with their solutions {---} a paradigm known as case-based reasoning (CBR). We propose a neuro-symbolic CBR approach (CBR-KBQA) for question answering over large knowledge bases. CBR-KBQA consists of a nonparametric memory that stores cases (question and logical forms) and a parametric model that can generate a logical form for a new question by retrieving cases that are relevant to it. On several KBQA datasets that contain complex questions, CBR-KBQA achieves competitive performance. For example, on the CWQ dataset, CBR-KBQA outperforms the current state of the art by 11{\%} on accuracy. Furthermore, we show that CBR-KBQA is capable of using new cases \textit{without} any further training: by incorporating a few human-labeled examples in the case memory, CBR-KBQA is able to successfully generate logical forms containing unseen KB entities as well as relations.",
}

RNG-KBQA: Generation augmented iterative ranking for knowledge base question answering.
摘要= "现有的KBQA方法，尽管在独立测试数据上取得了强大的性能，但在泛化涉及未见过的KB模式项的问题时往往很困难。之前基于排名的方法在泛化方面取得了一些成功，但存在覆盖率问题。本文提出RnG-KBQA，一种KBQA的排序和生成方法，用生成模型解决了覆盖率问题，同时保持了强大的泛化能力。该方法首先使用对比排序器对通过搜索知识图谱获得的一组候选逻辑形式进行排序。然后，它引入了一个以问题和排名靠前的候选人为条件的定制生成模型，以组成最终的逻辑形式。在GrailQA和WebQSP数据集上取得了新的最先进的结果。特别是，所提出方法在GrailQA排行榜上以很大的优势超过了之前的最先进方法。此外，RnG-KBQA在流行的WebQSP基准测试中优于之前的所有方法，甚至包括那些使用oracle实体链接的方法。实验结果证明了排名和生成之间相互作用的有效性，这导致了所提出方法在所有环境下的优越性能，在零样本泛化方面有特别强的改进。”
@inproceedings{Ye-Xi-ACL-2022-RNG-KBQA,
    title = "{RNG}-{KBQA}: Generation Augmented Iterative Ranking for Knowledge Base Question Answering",
    author = "Ye, Xi  and
      Yavuz, Semih  and
      Hashimoto, Kazuma  and
      Zhou, Yingbo  and
      Xiong, Caiming",
    editor = "Muresan, Smaranda  and
      Nakov, Preslav  and
      Villavicencio, Aline",
    booktitle = "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = may,
    year = "2022",
    address = "Dublin, Ireland",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.acl-long.417",
    doi = "10.18653/v1/2022.acl-long.417",
    pages = "6032--6043",
    abstract = "Existing KBQA approaches, despite achieving strong performance on i.i.d. test data, often struggle in generalizing to questions involving unseen KB schema items. Prior ranking-based approaches have shown some success in generalization, but suffer from the coverage issue. We present RnG-KBQA, a Rank-and-Generate approach for KBQA, which remedies the coverage issue with a generation model while preserving a strong generalization capability. Our approach first uses a contrastive ranker to rank a set of candidate logical forms obtained by searching over the knowledge graph. It then introduces a tailored generation model conditioned on the question and the top-ranked candidates to compose the final logical form. We achieve new state-of-the-art results on GrailQA and WebQSP datasets. In particular, our method surpasses the prior state-of-the-art by a large margin on the GrailQA leaderboard. In addition, RnG-KBQA outperforms all prior approaches on the popular WebQSP benchmark, even including the ones that use the oracle entity linking. The experimental results demonstrate the effectiveness of the interplay between ranking and generation, which leads to the superior performance of our proposed approach across all settings with especially strong improvements in zero-shot generalization.",
}

Program transfer for answering complex questions over knowledge bases.
摘要= "知识库上回答复杂问题的程序归纳旨在将问题分解为一个多步骤的程序，在知识库上执行该程序即可得到最终的答案。学习归纳程序依赖于给定知识库的大量并行问题-程序对。但是，大部分KBs缺少黄金节目注释，学习起来很困难。本文提出了程序迁移的方法，旨在利用资源丰富的KBs上有价值的程序注释作为外部监督信号，以辅助对缺乏程序注释的低资源KBs的程序归纳。针对程序迁移问题，本文设计了一种新的两阶段解析框架，并提出了一种有效的本体引导剪枝策略。首先，sketch解析器将问题转换为高级程序草图，即函数的组合;其次，给定问题和草图，参数解析器从KB中搜索详细的参数以查找函数。在搜索过程中，结合知识库本体对搜索空间进行剪枝。在ComplexWebQuestions和WebQuestionSP数据集上的实验结果表明，该方法的性能明显优于SOTA方法，验证了程序迁移和框架的有效性。我们的代码和数据集可以从\url{https://github.com/THU-KEG/ProgramTransfer}获得。”
@inproceedings{Caoshulin-ACL-2022-Program-Transfer,
    title = "Program Transfer for Answering Complex Questions over Knowledge Bases",
    author = "Cao, Shulin  and
      Shi, Jiaxin  and
      Yao, Zijun  and
      Lv, Xin  and
      Yu, Jifan  and
      Hou, Lei  and
      Li, Juanzi  and
      Liu, Zhiyuan  and
      Xiao, Jinghui",
    editor = "Muresan, Smaranda  and
      Nakov, Preslav  and
      Villavicencio, Aline",
    booktitle = "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = may,
    year = "2022",
    address = "Dublin, Ireland",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.acl-long.559",
    doi = "10.18653/v1/2022.acl-long.559",
    pages = "8128--8140",
    abstract = "Program induction for answering complex questions over knowledge bases (KBs) aims to decompose a question into a multi-step program, whose execution against the KB produces the final answer. Learning to induce programs relies on a large number of parallel question-program pairs for the given KB. However, for most KBs, the gold program annotations are usually lacking, making learning difficult. In this paper, we propose the approach of program transfer, which aims to leverage the valuable program annotations on the rich-resourced KBs as external supervision signals to aid program induction for the low-resourced KBs that lack program annotations. For program transfer, we design a novel two-stage parsing framework with an efficient ontology-guided pruning strategy. First, a sketch parser translates the question into a high-level program sketch, which is the composition of functions. Second, given the question and sketch, an argument parser searches the detailed arguments from the KB for functions. During the searching, we incorporate the KB ontology to prune the search space. The experiments on ComplexWebQuestions and WebQuestionSP show that our method outperforms SOTA methods significantly, demonstrating the effectiveness of program transfer and our framework. Our codes and datasets can be obtained from \url{https://github.com/THU-KEG/ProgramTransfer}.",
}

TIARA: Multi-grained retrieval for robust question answering over large knowledge base.
摘要= "预训练语言模型(plm)在多种场景中显示了其有效性。然而，KBQA仍然具有挑战性，特别是在覆盖率和泛化设置方面。这是由于两个主要因素:i)理解问题的语义和知识库中的相关知识;Ii)生成语义和语法都正确的可执行逻辑形式。本文提出了一种新的KBQA模型TIARA，通过应用多粒度检索来解决这些问题，以帮助PLM专注于最相关的KB上下文，即实体、示例逻辑形式和模式项。此外，采用约束译码来控制输出空间，减少生成误差。在重要基准上的实验证明了所提出方法的有效性。TIARA比之前的SOTA(包括那些使用plm或oracle实体注释的SOTA)在GrailQA和WebQuestionsSP上分别至少提高了4.1和1.1 F1分。特别是在GrailQA上，TIARA在所有类别中都优于之前的模型，在零样本泛化方面提高了4.7 F1分。”
@inproceedings{Shu-Yiheng-EMNLP-2022-TIARA,
    title = "{TIARA}: Multi-grained Retrieval for Robust Question Answering over Large Knowledge Base",
    author = {Shu, Yiheng  and
      Yu, Zhiwei  and
      Li, Yuhan  and
      Karlsson, B{\"o}rje  and
      Ma, Tingting  and
      Qu, Yuzhong  and
      Lin, Chin-Yew},
    editor = "Goldberg, Yoav  and
      Kozareva, Zornitsa  and
      Zhang, Yue",
    booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing",
    month = dec,
    year = "2022",
    address = "Abu Dhabi, United Arab Emirates",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.emnlp-main.555",
    doi = "10.18653/v1/2022.emnlp-main.555",
    pages = "8108--8121",
    abstract = "Pre-trained language models (PLMs) have shown their effectiveness in multiple scenarios. However, KBQA remains challenging, especially regarding coverage and generalization settings. This is due to two main factors: i) understanding the semantics of both questions and relevant knowledge from the KB; ii) generating executable logical forms with both semantic and syntactic correctness. In this paper, we present a new KBQA model, TIARA, which addresses those issues by applying multi-grained retrieval to help the PLM focus on the most relevant KB context, viz., entities, exemplary logical forms, and schema items. Moreover, constrained decoding is used to control the output space and reduce generation errors. Experiments over important benchmarks demonstrate the effectiveness of our approach. TIARA outperforms previous SOTA, including those using PLMs or oracle entity annotations, by at least 4.1 and 1.1 F1 points on GrailQA and WebQuestionsSP, respectively. Specifically on GrailQA, TIARA outperforms previous models in all categories, with an improvement of 4.7 F1 points in zero-shot generalization.",
}

ArcaneQA: Dynamic program induction and contextualized encoding for knowledge base question answering.
摘要= "知识库问答(KBQA)对语义解析研究提出了独特的挑战，这是因为两个相互交织的挑战:庞大的搜索空间和模式链接中的歧义性。传统的基于排序的KBQA模型依赖于候选人枚举步骤来减少搜索空间，在预测复杂查询时灵活性较差，且运行时间不切实际。本文提出ArcaneQA，一种新的基于生成的模型，在统一框架中通过两个相互促进的成分解决了大搜索空间和模式链接挑战:用于解决大搜索空间的动态程序归纳和用于模式链接的动态上下文编码。在多个流行的KBQA数据集上的实验结果表明，ArcaneQA在有效性和效率方面具有很强的竞争力。
@inproceedings{Gu-Yu-COLING-2022-ArcaneQA,
    title = "{A}rcane{QA}: Dynamic Program Induction and Contextualized Encoding for Knowledge Base Question Answering",
    author = "Gu, Yu  and
      Su, Yu",
    editor = "Calzolari, Nicoletta  and
      Huang, Chu-Ren  and
      Kim, Hansaem  and
      Pustejovsky, James  and
      Wanner, Leo  and
      Choi, Key-Sun  and
      Ryu, Pum-Mo  and
      Chen, Hsin-Hsi  and
      Donatelli, Lucia  and
      Ji, Heng  and
      Kurohashi, Sadao  and
      Paggio, Patrizia  and
      Xue, Nianwen  and
      Kim, Seokhwan  and
      Hahm, Younggyun  and
      He, Zhong  and
      Lee, Tony Kyungil  and
      Santus, Enrico  and
      Bond, Francis  and
      Na, Seung-Hoon",
    booktitle = "Proceedings of the 29th International Conference on Computational Linguistics",
    month = oct,
    year = "2022",
    address = "Gyeongju, Republic of Korea",
    publisher = "International Committee on Computational Linguistics",
    url = "https://aclanthology.org/2022.coling-1.148",
    pages = "1718--1731",
    abstract = "Question answering on knowledge bases (KBQA) poses a unique challenge for semantic parsing research due to two intertwined challenges: large search space and ambiguities in schema linking. Conventional ranking-based KBQA models, which rely on a candidate enumeration step to reduce the search space, struggle with flexibility in predicting complicated queries and have impractical running time. In this paper, we present ArcaneQA, a novel generation-based model that addresses both the large search space and the schema linking challenges in a unified framework with two mutually boosting ingredients: dynamic program induction for tackling the large search space and dynamic contextualized encoding for schema linking. Experimental results on multiple popular KBQA datasets demonstrate the highly competitive performance of ArcaneQA in both effectiveness and efficiency.",
}

Logical form generation via multi-task learning for complex question answering over knowledge bases.
摘要= "针对复杂问题的知识库问答(KBQA)是自然语言处理中一项具有挑战性的任务。最近，基于生成的方法将自然语言问题翻译为可执行的逻辑形式，取得了很好的性能。这些方法使用辅助信息来增强未见过的KB项目或新组合的问题的逻辑形式的生成，但引入的噪声也会导致更多不正确的结果。本文提出GMT-KBQA，一种基于多任务学习的KBQA方法，以更好地检索和利用辅助信息。GMT-KBQA首先通过密集检索获得候选实体和关系，然后引入一个联合学习实体消歧、关系分类和逻辑形式生成的多任务模型。实验结果表明，GMT-KBQA在ComplexWebQuestions和WebQuestionsSP数据集上均取得了较好的效果。此外，详细的评估表明，GMT-KBQA受益于辅助任务，具有较强的泛化能力。
@inproceedings{Hu-Xixin-COLING-2022-Logical-Form-Generation,
    title = "Logical Form Generation via Multi-task Learning for Complex Question Answering over Knowledge Bases",
    author = "Hu, Xixin  and
      Wu, Xuan  and
      Shu, Yiheng  and
      Qu, Yuzhong",
    editor = "Calzolari, Nicoletta  and
      Huang, Chu-Ren  and
      Kim, Hansaem  and
      Pustejovsky, James  and
      Wanner, Leo  and
      Choi, Key-Sun  and
      Ryu, Pum-Mo  and
      Chen, Hsin-Hsi  and
      Donatelli, Lucia  and
      Ji, Heng  and
      Kurohashi, Sadao  and
      Paggio, Patrizia  and
      Xue, Nianwen  and
      Kim, Seokhwan  and
      Hahm, Younggyun  and
      He, Zhong  and
      Lee, Tony Kyungil  and
      Santus, Enrico  and
      Bond, Francis  and
      Na, Seung-Hoon",
    booktitle = "Proceedings of the 29th International Conference on Computational Linguistics",
    month = oct,
    year = "2022",
    address = "Gyeongju, Republic of Korea",
    publisher = "International Committee on Computational Linguistics",
    url = "https://aclanthology.org/2022.coling-1.145",
    pages = "1687--1696",
    abstract = "Question answering over knowledge bases (KBQA) for complex questions is a challenging task in natural language processing. Recently, generation-based methods that translate natural language questions to executable logical forms have achieved promising performance. These methods use auxiliary information to augment the logical form generation of questions with unseen KB items or novel combinations, but the noise introduced can also leads to more incorrect results. In this work, we propose GMT-KBQA, a Generation-based KBQA method via Multi-Task learning, to better retrieve and utilize auxiliary information. GMT-KBQA first obtains candidate entities and relations through dense retrieval, and then introduces a multi-task model which jointly learns entity disambiguation, relation classification, and logical form generation. Experimental results show that GMT-KBQA achieves state-of-the-art results on both ComplexWebQuestions and WebQuestionsSP datasets. Furthermore, the detailed evaluation demonstrates that GMT-KBQA benefits from the auxiliary tasks and has a strong generalization capability.",
}

Uni-parser: Unified semantic parser for question answering on knowledge base and database.
abstract = "将自然语言问题解析为可执行的逻辑形式是对结构化数据(如知识库或数据库)进行问答的一种有用和可解释的方法。然而，现有的语义分析方法无法同时适应两种模态，因为它们受到候选逻辑形式的指数级增长的影响，并且难以推广到未见过的数据。本文提出Uni-Parser，一个面向知识库和数据库的问答(QA)的统一语义解析器。我们将原语(KB中的关系和实体，DB中的表名、列名和单元格值)定义为框架中的基本元素。基元的数量仅以KB和DB为单位的检索关系的数量的线性速度增长，这使得我们无法使用指数逻辑形式的候选。利用生成器，通过使用不同的操作(例如select、where、count)更改和组合排名靠前的基元，来预测最终的逻辑形式。通过对比基元排序器充分剪枝搜索空间，生成器被赋予捕捉基元组合的能力，增强了其泛化能力。在多个KB和DB QA基准上取得了有竞争力的结果，效率更高，特别是在组合和零样本设置中。”
@inproceedings{Liu-Ye-EMNLP-2022-Uni-Parser,
    title = "Uni-Parser: Unified Semantic Parser for Question Answering on Knowledge Base and Database",
    author = "Liu, Ye  and
      Yavuz, Semih  and
      Meng, Rui  and
      Radev, Dragomir  and
      Xiong, Caiming  and
      Zhou, Yingbo",
    editor = "Goldberg, Yoav  and
      Kozareva, Zornitsa  and
      Zhang, Yue",
    booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing",
    month = dec,
    year = "2022",
    address = "Abu Dhabi, United Arab Emirates",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.emnlp-main.605",
    doi = "10.18653/v1/2022.emnlp-main.605",
    pages = "8858--8869",
    abstract = "Parsing natural language questions into executable logical forms is a useful and interpretable way to perform question answering on structured data such as knowledge bases (KB) or databases (DB). However, existing approaches on semantic parsing cannot adapt to both modalities, as they suffer from the exponential growth of the logical form candidates and can hardly generalize to unseen data. In this work, we propose Uni-Parser, a unified semantic parser for question answering (QA) on both KB and DB. We define the primitive (relation and entity in KB, and table name, column name and cell value in DB) as the essential element in our framework. The number of primitives grows only at a linear rate to the number of retrieved relations in KB and DB, preventing us from exponential logic form candidates. We leverage the generator to predict final logical forms by altering and composing top-ranked primitives with different operations (e.g. select, where, count). With sufficiently pruned search space by a contrastive primitive ranker, the generator is empowered to capture the composition of primitives enhancing its generalization ability. We achieve competitive results on multiple KB and DB QA benchmarks with more efficiency, especially in the compositional and zero-shot settings.",
}


UnifiedSKG: Unifying and multi-tasking structured knowledge grounding with text-to-text language models.
摘要= "结构化知识基础(Structured knowledge grounding, SKG)利用结构化知识来完成用户请求，如基于数据库的语义解析和基于知识库的问答。由于SKG任务的输入和输出是异构的，不同的社团对其进行了独立的研究，这限制了对SKG的系统和兼容的研究。本文通过提出UnifiedSKG框架克服了这一限制，将21个SKG任务统一为文本到文本格式，旨在促进系统的SKG研究，而不是只局限于单个任务、领域或数据集。使用UnifiedSKG对不同大小的T5进行基准测试，结果表明，T5只要在必要时进行简单修改，就可以在几乎所有21个任务上实现最先进的性能。本文进一步证明，多任务前缀调优提高了大多数任务的性能，在很大程度上提高了整体性能。UnifiedSKG还促进了对零样本和少样本学习的研究，T0、GPT-3和Codex在SKG的零样本和少样本学习中遇到了困难。还使用UnifiedSKG在SKG任务中对结构化知识编码变体进行了一系列控制实验。UnifiedSKG很容易扩展到更多的任务，它的开源地址是\url{https://github.com/hkunlp/unifiedskg}。”
@inproceedings{Xie-Tianbao-EMNLP-2022-UnifiedSKG,
    title = "{U}nified{SKG}: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models",
    author = "Xie, Tianbao  and
      Wu, Chen Henry  and
      Shi, Peng  and
      Zhong, Ruiqi  and
      Scholak, Torsten  and
      Yasunaga, Michihiro  and
      Wu, Chien-Sheng  and
      Zhong, Ming  and
      Yin, Pengcheng  and
      Wang, Sida I.  and
      Zhong, Victor  and
      Wang, Bailin  and
      Li, Chengzu  and
      Boyle, Connor  and
      Ni, Ansong  and
      Yao, Ziyu  and
      Radev, Dragomir  and
      Xiong, Caiming  and
      Kong, Lingpeng  and
      Zhang, Rui  and
      Smith, Noah A.  and
      Zettlemoyer, Luke  and
      Yu, Tao",
    editor = "Goldberg, Yoav  and
      Kozareva, Zornitsa  and
      Zhang, Yue",
    booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing",
    month = dec,
    year = "2022",
    address = "Abu Dhabi, United Arab Emirates",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.emnlp-main.39",
    doi = "10.18653/v1/2022.emnlp-main.39",
    pages = "602--631",
    abstract = "Structured knowledge grounding (SKG) leverages structured knowledge to complete user requests, such as semantic parsing over databases and question answering over knowledge bases. Since the inputs and outputs of SKG tasks are heterogeneous, they have been studied separately by different communities, which limits systematic and compatible research on SKG. In this paper, we overcome this limitation by proposing the UnifiedSKG framework, which unifies 21 SKG tasks into a text-to-text format, aiming to promote systematic SKG research, instead of being exclusive to a single task, domain, or dataset. We use UnifiedSKG to benchmark T5 with different sizes and show that T5, with simple modifications when necessary, achieves state-of-the-art performance on almost all of the 21 tasks. We further demonstrate that multi-task prefix-tuning improves the performance on most tasks, largely improving the overall performance. UnifiedSKG also facilitates the investigation of zero-shot and few-shot learning, and we show that T0, GPT-3, and Codex struggle in zero-shot and few-shot learning for SKG. We also use UnifiedSKG to conduct a series of controlled experiments on structured knowledge encoding variants across SKG tasks. UnifiedSKG is easily extensible to more tasks, and it is open-sourced at \url{https://github.com/hkunlp/unifiedskg}.",
}

DecAF: Joint decoding of answers and logical forms for question answering over knowledge bases.
摘要= "知识库问答(KBs)旨在回答知识库中包含事实信息的自然语言问题，如实体和关系等。以前的方法要么生成可在KBs上执行的逻辑形式以获得最终答案，要么直接预测答案。实验结果表明，前者往往能生成更准确的答案，但由于生成的逻辑形式存在潜在的语法和语义错误，因此存在非执行问题。本文提出一种新的框架DecAF，联合生成逻辑形式和直接答案，然后结合它们的优点来获得最终答案。此外，与以往大多数方法不同的是，DecAF基于简单的自由文本检索，不依赖于任何实体链接工具——这种简化简化了它对不同数据集的适应。DecAF在WebQSP、FreebaseQA和GrailQA基准上实现了最先进的准确性，同时在ComplexWebQuestions基准上获得了有竞争力的结果。”
@inproceedings{Yu-Donghan-ICLR-2023-DecAF,
    title={Dec{AF}: Joint Decoding of Answers and Logical Forms for Question Answering over Knowledge Bases},
    author={Donghan Yu and Sheng Zhang and Patrick Ng and Henghui Zhu and Alexander Hanbo Li and Jun Wang and Yiqun Hu and William Yang Wang and Zhiguo Wang and Bing Xiang},
    booktitle={The Eleventh International Conference on Learning Representations },
    year={2023},
    url={https://openreview.net/forum?id=XHc5zRPxqV9},
    abstract = "Question answering over knowledge bases (KBs) aims to answer natural language questions with factual information such as entities and relations in KBs. Previous methods either generate logical forms that can be executed over KBs to obtain final answers or predict answers directly. Empirical results show that the former often produces more accurate answers, but it suffers from  non-execution issues due to potential syntactic and semantic errors in the generated logical forms. In this work, we propose a novel framework DecAF that jointly generates both logical forms and direct answers, and then combines the merits of them to get the final answers. Moreover, different from most of the previous methods, DecAF is based on simple free-text retrieval without relying on any entity linking tools --- this simplification eases its adaptation to different datasets. DecAF achieves new state-of-the-art accuracy on WebQSP, FreebaseQA, and GrailQA benchmarks, while getting competitive results on the ComplexWebQuestions benchmark."
}

FC-KBQA: A fine-to-coarse composition framework for knowledge base question answering.
摘要= " KBQA上的泛化问题引起了人们的广泛关注。现有研究存在由粗粒度的逻辑表达式建模纠缠带来的泛化问题，或由实际KBs中断开连接的类和关系的细粒度建模带来的不可执行问题。本文提出了一种由细到粗的KBQA组合框架(FC-KBQA)，以保证逻辑表达式的泛化能力和可执行性。FC-KBQA的主要思想是从知识库中提取相关的细粒度知识组件，并将其重新形式化为中粒度知识对，以生成最终的逻辑表达式。FC-KBQA在GrailQA和WebQSP上获得了最先进的性能，运行速度比基线快4倍。我们的代码现在可以在GitHub \url{https://github.com/RUCKBReasoning/FC-KBQA}上找到。”
@inproceedings{Zhang-Lingxi-ACL-2023-FC-KBQA,
    title = "{FC}-{KBQA}: A Fine-to-Coarse Composition Framework for Knowledge Base Question Answering",
    author = "Zhang, Lingxi  and
      Zhang, Jing  and
      Wang, Yanling  and
      Cao, Shulin  and
      Huang, Xinmei  and
      Li, Cuiping  and
      Chen, Hong  and
      Li, Juanzi",
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.acl-long.57",
    doi = "10.18653/v1/2023.acl-long.57",
    pages = "1002--1017",
    abstract = "The generalization problem on KBQA has drawn considerable attention. Existing research suffers from the generalization issue brought by the entanglement in the coarse-grained modeling of the logical expression, or inexecutability issues due to the fine-grained modeling of disconnected classes and relations in real KBs. We propose a Fine-to-Coarse Composition framework for KBQA (FC-KBQA) to both ensure the generalization ability and executability of the logical expression. The main idea of FC-KBQA is to extract relevant fine-grained knowledge components from KB and reformulate them into middle-grained knowledge pairs for generating the final logical expressions. FC-KBQA derives new state-of-the-art performance on GrailQA and WebQSP, and runs 4 times faster than the baseline. Our code is now available at GitHub \url{https://github.com/RUCKBReasoning/FC-KBQA}.",
}

# -------------- LLM+SP -------------- #
Don’t Generate, Discriminate: A Proposal for Grounding Language Models to Real-World Environments
@inproceedings{Gu-Yu-ACL-2023-LLM+SP,
    title = "Don{'}t Generate, Discriminate: A Proposal for Grounding Language Models to Real-World Environments",
    author = "Gu, Yu  and
      Deng, Xiang  and
      Su, Yu",
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.acl-long.270",
    doi = "10.18653/v1/2023.acl-long.270",
    pages = "4928--4949",
    abstract = "A key missing capacity of current language models (LMs) is grounding to real-world environments. Most existing work for grounded language understanding uses LMs to directly generate plans that can be executed in the environment to achieve the desired effects. It thereby casts the burden of ensuring grammaticality, faithfulness, and controllability all on the LMs. We propose Pangu, a generic framework for grounded language understanding that capitalizes on the discriminative ability of LMs instead of their generative ability. Pangu consists of a symbolic agent and a neural LM working in a concerted fashion: The agent explores the environment to incrementally construct valid plans, and the LM evaluates the plausibility of the candidate plans to guide the search process. A case study on the challenging problem of knowledge base question answering (KBQA), which features a massive environment, demonstrates the remarkable effectiveness and flexibility of Pangu: A BERT-base LM is sufficient for setting a new record on standard KBQA datasets, and larger LMs further bring substantial gains.Pangu also enables, for the first time, effective few-shot in-context learning for KBQA with large LMs such as Codex.",
}

ChatKBQA- A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models
摘要:"知识库问答(Knowledge Knowledge Answering, KBQA)旨在从大规模知识库(Knowledge Base, KBs)中获取自然语言问题的答案，其研究内容一般分为知识检索和语义解析两个部分。然而，仍然存在三个核心挑战，包括低效的知识检索、影响语义解析的检索错误以及以往KBQA方法的复杂性。在大型语言模型(llm)时代，本文提出ChatKBQA，一种新的先生成后检索的KBQA框架，建立在微调开源llm上，如Llama-2、ChatGLM2和Baichuan2。ChatKBQA提出先生成经过微调的llm逻辑形式，然后通过无监督检索方法检索和替换实体和关系，更直接地改进了生成和检索。实验结果表明，ChatKBQA在标准KBQA数据集WebQSP和ComplexWebQuestions (CWQ)上取得了最新的性能。这项工作还提供了一种新的范式，将llm与知识图谱(KGs)相结合，以实现可解释的和知识所需的问答。我们的代码是公开的。
@misc{Haoran-Luo-arXiv-2023-ChatKBQA,
      title={ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models}, 
      author={Haoran Luo and Haihong E and Zichen Tang and Shiyao Peng and Yikai Guo and Wentai Zhang and Chenghao Ma and Guanting Dong and Meina Song and Wei Lin},
      year={2023},
      eprint={2310.08975},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      abstract="Knowledge Base Question Answering (KBQA) aims to derive answers to natural language questions over large-scale knowledge bases (KBs), which are generally divided into two research components: knowledge retrieval and semantic parsing. However, three core challenges remain, including inefficient knowledge retrieval, retrieval errors adversely affecting semantic parsing, and the complexity of previous KBQA methods. In the era of large language models (LLMs), we introduce ChatKBQA, a novel generate-then-retrieve KBQA framework built on fine-tuning open-source LLMs such as Llama-2, ChatGLM2 and Baichuan2. ChatKBQA proposes generating the logical form with fine-tuned LLMs first, then retrieving and replacing entities and relations through an unsupervised retrieval method, which improves both generation and retrieval more straightforwardly. Experimental results reveal that ChatKBQA achieves new state-of-the-art performance on standard KBQA datasets, WebQSP, and ComplexWebQuestions (CWQ). This work also provides a new paradigm for combining LLMs with knowledge graphs (KGs) for interpretable and knowledge-required question answering. Our code is publicly available."
}

其他
Flexkbqa 

# -------------- Datasets -------------- #

webqsp

cwq

kqapro
abstract = "基于知识库的复杂问答(Complex KBQA)是一个具有挑战性的问题，因为它需要多种组合推理能力，如多跳推理、属性比较、集合操作等。现有的基准测试存在一些缺点，限制了复杂KBQA的开发:1)只提供没有显式推理过程的QA对;2)问题的多样性和规模较差。本文提出KQA Pro，一个用于复杂KBQA的数据集，包括大约12万个不同的自然语言问题。本文提出一种可组合和可解释的程序设计语言KoPL来表示复杂问题的推理过程。对于每个问题，我们都提供了相应的KoPL程序和SPARQL查询，使得KQA Pro可以同时服务于KBQA和语义解析任务。实验结果表明，当前最先进的KBQA方法在KQA Pro数据集上不能取得令人满意的效果，这表明KQA Pro是具有挑战性的，复杂的KBQA还需要进一步的研究。将KQA Pro作为测试多种推理技能的诊断数据集，对现有模型进行了彻底的评估，并讨论了复杂KBQA的进一步方向。我们的代码和数据集可以从\url{https://github.com/shijx12/KQAPro_Baselines}获得。”
@inproceedings{Cao-Shulin-ACL-2022-KQAPro,
    title = "{KQA} Pro: A Dataset with Explicit Compositional Programs for Complex Question Answering over Knowledge Base",
    author = "Cao, Shulin  and
      Shi, Jiaxin  and
      Pan, Liangming  and
      Nie, Lunyiu  and
      Xiang, Yutong  and
      Hou, Lei  and
      Li, Juanzi  and
      He, Bin  and
      Zhang, Hanwang",
    editor = "Muresan, Smaranda  and
      Nakov, Preslav  and
      Villavicencio, Aline",
    booktitle = "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = may,
    year = "2022",
    address = "Dublin, Ireland",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.acl-long.422",
    doi = "10.18653/v1/2022.acl-long.422",
    pages = "6101--6119",
    abstract = "Complex question answering over knowledge base (Complex KBQA) is challenging because it requires various compositional reasoning capabilities, such as multi-hop inference, attribute comparison, set operation, etc. Existing benchmarks have some shortcomings that limit the development of Complex KBQA: 1) they only provide QA pairs without explicit reasoning processes; 2) questions are poor in diversity or scale. To this end, we introduce KQA Pro, a dataset for Complex KBQA including around 120K diverse natural language questions. We introduce a compositional and interpretable programming language KoPL to represent the reasoning process of complex questions. For each question, we provide the corresponding KoPL program and SPARQL query, so that KQA Pro can serve for both KBQA and semantic parsing tasks. Experimental results show that state-of-the-art KBQA methods cannot achieve promising results on KQA Pro as on current datasets, which suggests that KQA Pro is challenging and Complex KBQA requires further research efforts. We also treat KQA Pro as a diagnostic dataset for testing multiple reasoning skills, conduct a thorough evaluation of existing models and discuss further directions for Complex KBQA. Our codes and datasets can be obtained from \url{https://github.com/shijx12/KQAPro_Baselines}.",
}

metaqa
众所周知，知识图(知识图谱)有助于问答(QA)任务，因为它提供了实体之间结构良好的关系信息，并允许人们进一步推断间接事实。然而，构建能够学习仅基于问题-答案对的知识图谱推理的问答系统具有挑战性。首先，当人们提出问题时，他们的表达是嘈杂的(例如，文本中的拼写错误，或发音变化)，这对于QA系统来说，将这些提到的实体匹配到知识图谱是很重要的。其次，许多问题需要在知识图谱上进行多跳逻辑推理才能检索到答案。为应对这些挑战，本文提出了一种新的、统一的深度学习架构和一种端到端的变分学习算法，可以处理问题中的噪声，并同时学习多跳推理。该方法在文献中最近的基准数据集上取得了最先进的性能。导出了一系列新的基准数据集，包括多跳推理问题、神经翻译模型转述的问题和人类声音的问题。该方法在所有这些具有挑战性的数据集上都产生了非常有希望的结果。
@inproceedings{Zhang-Yuyu-AAAI-2018-MetaQA,
    author = {Zhang, Yuyu and Dai, Hanjun and Kozareva, Zornitsa and Smola, Alexander J. and Song, Le},
    title = {Variational reasoning for question answering with knowledge graph},
    year = {2018},
    isbn = {978-1-57735-800-8},
    publisher = {AAAI Press},
    abstract = {Knowledge graph (KG) is known to be helpful for the task of question answering (QA), since it provides well-structured relational information between entities, and allows one to further infer indirect facts. However, it is challenging to build QA systems which can learn to reason over knowledge graphs based on question-answer pairs alone. First, when people ask questions, their expressions are noisy (for example, typos in texts, or variations in pronunciations), which is non-trivial for the QA system to match those mentioned entities to the knowledge graph. Second, many questions require multi-hop logic reasoning over the knowledge graph to retrieve the answers. To address these challenges, we propose a novel and unified deep learning architecture, and an end-to-end variational learning algorithm which can handle noise in questions, and learn multi-hop reasoning simultaneously. Our method achieves state-of-the-art performance on a recent benchmark dataset in the literature. We also derive a series of new benchmark datasets, including questions for multi-hop reasoning, questions paraphrased by neural translation model, and questions in human voice. Our method yields very promising results on all these challenging datasets.},
    booktitle = {Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence},
    articleno = {745},
    numpages = {8},
    location = {New Orleans, Louisiana, USA},
    series = {AAAI'18/IAAI'18/EAAI'18}
}


# -------------- Baseline -------------- #

